Baselines for Joint-Action Reinforcement Learning of Coordination in Cooperative Multi-agent Systems
نویسندگان
چکیده
We report on an investigation of reinforcement learning techniques for the learning of coordination in cooperative multiagent systems. Specifically, we focus on a novel action selection strategy for Q-learning (Watkins 1989). The new technique is applicable to scenarios where mutual observation of actions is not possible. To date, reinforcement learning approaches for such independent agents did not guarantee convergence to the optimal joint action in scenarios with high miscoordination costs. We improve on previous results (Claus & Boutilier 1998) by demonstrating empirically that our extension causes the agents to converge almost always to the optimal joint action even in these difficult cases.
منابع مشابه
Improving on the reinforcement learning of coordination in cooperative multi-agent systems
We report on an investigation of reinforcement learning techniques for the learning of coordination in cooperative multiagent systems. These techniques are variants of Q-learning (Watkins, 1989) that are applicable to scenarios where mutual observation of actions is not possible. To date, reinforcement learning approaches for such independent agents did not guarantee convergence to the optimal ...
متن کاملVoltage Coordination of FACTS Devices in Power Systems Using RL-Based Multi-Agent Systems
This paper describes how multi-agent system technology can be used as the underpinning platform for voltage control in power systems. In this study, some FACTS (flexible AC transmission systems) devices are properly designed to coordinate their decisions and actions in order to provide a coordinated secondary voltage control mechanism based on multi-agent theory. Each device here is modeled as ...
متن کاملMultiagent Coordination in Cooperative Q-learning Systems
Many reinforcement learning architectures fail to learn optimal group behaviors in the multiagent domain. Although these coordination difficulties are often attributed to the non-Markovian environment created by the gradually-changing policies of concurrently learning agents, a careful analysis of the situation reveals an underlying problem structure which can cause suboptimal group policies ev...
متن کاملReinforcement Social Learning of Coordination in Networked Cooperative Multiagent Systems
The problem of coordination in cooperative multiagent systems has been widely studied in the literature. In practical complex environments, the interactions among agents are usually regulated by their underlying network topology, which, however, has not been taken into consideration in previous work. To this end, we firstly investigate the multiagent coordination problems in cooperative environ...
متن کاملLearning to Coordinate Using Commitment Sequences in Cooperative Multi-agent Systems
We report on an investigation of the learning of coordination in cooperative multiagent systems. Specifically, we study solutions that are applicable to independent agents, i.e., agents that do not observe one another’s actions and do not explicitly communicate with each other. In previously published work (Kapetanakis and Kudenko, 2002) we have presented a reinforcement learning approach that ...
متن کامل